Skip to content

fix(ci): stop uv.lock PyPI hashes failing the secret scan - #1159

Closed
groupthinking wants to merge 1 commit into
mainfrom
groupthinking-fix-gitleaks-lockfile-false-positive
Closed

fix(ci): stop uv.lock PyPI hashes failing the secret scan#1159
groupthinking wants to merge 1 commit into
mainfrom
groupthinking-fix-gitleaks-lockfile-false-positive

Conversation

@groupthinking

@groupthinking groupthinking commented Aug 1, 2026

Copy link
Copy Markdown
Owner

Canonical issue

Fixes #1158

Outcome

The Secret Scan / gitleaks (working tree) job stops failing on every pull
request in the repository.

gitleaks flags one line of uv.lock that contains no secret:

Finding:     30/4b/90c937815137d43ce71ba043cd3566221e9df6b9c805f24b5d138c9d40a7
Rule:        square-access-token
File:        uv.lock
Line:        5129
Entropy:     3.884

That string is the PyPI content-hash path segment of the parso 0.8.7 sdist URL.
It is high-entropy by construction and matches the square-access-token
pattern by coincidence. Because uv.lock is committed and unrelated to any
individual change, every PR inherits the failure — the job has been red
repo-wide, which is the worst possible state for a secret scanner: a check
nobody can distinguish a real finding in.

After this change the scan reports no leaks found on a clean tree, so a
future red result is signal again.

Scope

  • Included: a narrowly targeted allowlist entry in .gitleaks.toml, and a test
    that pins the properties which make it narrow.
  • Explicitly excluded: uv.lock itself (unchanged), the gitleaks version pin,
    .github/workflows/secret-scan.yml, and the default rule set — useDefault
    stays on.

What was deliberately not done: the obvious fix is
paths = ['''uv\.lock''']. That is wrong. A uv lockfile can legitimately carry
a private index URL with inline credentials
(https://user:token@pypi.internal/simple/...), which is exactly the leak this
job exists to catch. Excluding the file by path would blind the scanner to the
one real secret it is most likely to find there. The allowlist is therefore
anchored to the benign pattern instead of to the file.

Risk

  • Risk level: low, but it is a change to a security control, so the narrowness
    matters more than the size of the diff.
  • Failure mode: an over-broad allowlist that suppresses a genuine secret. Two
    properties bound it, and both are asserted by tests: the regex is anchored to
    the literal host https://files\.pythonhosted\.org/packages/, so it cannot
    match a credential on any other host; and it is a regexes entry, not a
    paths entry, so no file is ever wholly exempt.
  • Rollback: revert this commit. The scanner returns to its previous behaviour —
    which is to say, red on every PR again.

Note on gitleaks semantics that shaped the implementation:

  • The pinned version is 8.18.4, which supports only a single [allowlist]
    block. [[allowlists]] with condition = "AND" arrived in 8.19. paths and
    regexes are therefore OR-ed and cannot be AND-combined, which is a second,
    independent reason the entry had to be pattern-anchored rather than
    path-plus-pattern.
  • regexTarget = "line" is load-bearing. The extracted secret is the bare hex
    segment and contains no host at all, so a host-anchored regex only matches
    when evaluated against the whole source line. Without it the allowlist
    silently does nothing.

Verification

Reproduced and verified with the same gitleaks version CI pins, 8.18.4,
installed locally, using the same command as the workflow
(gitleaks detect --no-git --config .gitleaks.toml --redact --verbose --exit-code 1):

before:  leaks found: 1
after:   no leaks found

Negative test — the part that matters. A suppression that merely makes the
scan green is worthless. To prove the allowlist is scoped to the benign pattern
and not to the file, the offending line was duplicated into uv.lock with the
host swapped to pypi.internal.example.com:

still flagged: leaks found: 1

The scanner still catches a same-shaped, high-entropy string on a different
host inside the very file being allowlisted. That is the property that would
have been destroyed by a path exclusion.

tests/unit/test_secret_scan_config.py5/5 pass
(PYTHONPATH=. python3 -m unittest tests.unit.test_secret_scan_config). It
asserts: useDefault remains enabled; no lockfile is path-allowlisted; the
regex retains both the host anchor and the /packages/ prefix; regexTarget
is line; and secret-scan.yml still passes --config .gitleaks.toml so the
config is actually consulted.

Mutation-tested three ways, each applied, run, and reverted:

CAUGHT   blanket uv.lock path exclusion
CAUGHT   host anchor removed from the regex
CAUGHT   regexTarget dropped (allowlist becomes inert)
  • Focused tests
  • Required CI — gitleaks (working tree) and agent-completion/truth-gate both pass on this head
  • Review threads resolved

Agent completion enforcement is red here, as it is on every open PR in the
repository, because its trust policy is unprovisioned. Surveyed and tracked
separately in #1160; it is unrelated to this diff.

Production evidence

Not applicable: no runtime surface changes. The diff is one CI configuration
file plus its test; no application code, dependency, or infrastructure is
touched, and uv.lock is byte-identical.

The evidence for a scanner change is the scanner's own behaviour, which is why
verification above is a real 8.18.4 run at the pinned version rather than a
reimplementation of the rule, and why it includes a negative case proving the
suppression still fails closed on a different host.

Notes for reviewers

Recommend merging this first of the currently open fixes. It unblocks the
secret scan for every PR in the repository, including #1154 and #1155, which are
cut from a main that predates it and stay red on this check until it lands.

Agent handoff

  • One canonical issue is linked
  • No competing PR implements the same issue
  • Acceptance criteria are satisfied
  • Required checks pass on the current head
  • Human decision is requested only for product, security, irreversible infrastructure, or production approval

`gitleaks (working tree)` failed on every pull request. uv records each
artifact as a PyPI download URL whose path embeds the artifact's own
content hash, and the segment for parso 0.8.7 has enough length and
entropy to match the default `square-access-token` rule. Nothing in a
pull request's diff could cause or clear it.

Allowlist the pattern by line, anchored on the public PyPI CDN host,
rather than excluding the lockfile by path. A path exclusion would also
hide a private index URL that embeds credentials inline, which is the
case this job exists to catch.

Verified with the pinned gitleaks 8.18.4: the working tree now reports
no leaks, and a copy of the same offending URL rehosted on
`pypi.internal.example.com` is still reported.

tests/unit/test_secret_scan_config.py pins the shape of the suppression
so it cannot be widened later: lockfiles stay scanned, the regex keeps
its host anchor and `/packages/` prefix, `regexTarget` stays `line`, and
the workflow keeps loading the config.

Fixes #1158

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 1, 2026 19:52
@vercel

vercel Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
v0-uvai Canceled Canceled v0 Aug 1, 2026 7:52pm

@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one) (1)
  • [‘architecture-gap’, ‘bug’, ‘ci-cd’, ‘ci/cd’, ‘copilot-rabbit’, ‘documentation’, ‘duplicate’, ‘enhancement’, ‘frontend’, ‘github_actions’, ‘good first issue’, ‘help wanted’, ‘high-priority’, ‘invalid’, ‘javascript’, ‘ml-model’, ‘needs-triage’, ‘pipeline-critical’, ‘placeholder-code’, ‘priority:high’, ‘python’, ‘python:uv’, ‘question’, ‘styling’, ‘tests’, ‘v0’]

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository YAML (base), Repository UI (inherited), Organization UI (inherited)

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 179342f5-ce5e-49eb-8520-0d882eaaf4df

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the python label Aug 1, 2026
@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Snapshot Warnings

⚠️: No snapshots were found for the head SHA 3bdc33e.
Ensure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice.

Scanned Files

None

@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown

Agent Completion Truth Gate: NOT_APPLICABLE

Evidence agrees.

Machine-readable verdict
{
  "details": {},
  "reasons": [],
  "verdict": "not_applicable"
}

Workflow evidence

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Narrows a Gitleaks false-positive suppression for public PyPI artifact hashes while retaining lockfile scanning.

Changes:

  • Adds a line-targeted PyPI CDN allowlist.
  • Adds regression tests for scan configuration and workflow integration.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
.gitleaks.toml Adds the PyPI suppression rule.
tests/unit/test_secret_scan_config.py Adds configuration regression guards.

Comment on lines +71 to +75
regexes = self.allowlist.get("regexes", [])
self.assertTrue(
any("files" in r and "pythonhosted" in r for r in regexes),
"expected an allowlist regex anchored on files.pythonhosted.org; "
f"got {regexes!r}",
Comment on lines +59 to +63
for lockfile in SCANNED_LOCKFILES:
for pattern in paths:
self.assertNotIn(
lockfile,
pattern,
Comment thread .gitleaks.toml
Comment on lines +11 to +12
regexTarget = "line"
regexes = [

@groupthinking groupthinking left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — verified locally at 3bdc33e

The core fix is correct and does what it claims. I checked out this head and confirmed:

  • tests/unit/test_secret_scan_config.py5/5 pass (PYTHONPATH=. python3 -m unittest tests.unit.test_secret_scan_config).
  • The required gitleaks (working tree) check is green on this head, alongside CodeQL, Trivy, bandit, python-safety, npm-audit, and both Security Scans. The only red status is a manually-canceled Vercel deployment (Canceled from the Vercel Dashboard) — infra noise, not this diff. The two red Agent completion enforcement runs are the repo-wide governance tooling that is failing across many PRs, not something in this change.

I could not re-run gitleaks 8.18.4 here (not installed in this sandbox), so I'm relying on the before/after + mutation evidence in the description for the scanner behavior itself, which is thorough.

The pattern-anchored-not-path-anchored decision is the right call, and the rationale in the description (a private index URL with inline credentials is exactly the secret this job should still catch) is exactly right.

On the three Copilot comments — all three are valid; I confirmed each against the code

  1. .gitleaks.toml:12 (most important). regexTarget = "line" on the top-level [allowlist] applies to every rule, so any finding on a line containing files.pythonhosted.org/packages/ is suppressed — not just square-access-token. In a committed uv.lock the practical blast radius is small (each line is a single artifact URL), but it is a genuine widening of a security control: a real token sharing a line with that URL would be hidden. Worth either scoping the exception to the square-access-token rule (noting the 8.18.4 single-[allowlist] constraint you already documented — rule-scoping needs [[rules.allowlists]], which is more involved) or consciously accepting the residual risk with a one-line note in the config comment so the decision is explicit.

  2. test_secret_scan_config.py:63assertNotIn("uv.lock", pattern) is a substring check, so a regex path exclusion like .*\.lock$ or ^uv[.]lock$ would exclude uv.lock while sailing past this guard. Compile each paths entry and assert it does not match each protected filename.

  3. test_secret_scan_config.py:75 — the host-anchor test asserts substrings are present in the regex, not that the regex rejects a foreign host. An alternation that also permits another host would still pass. Exercise the configured regex against both the public URL and an identical private-host URL so that widening the suppression fails the suite.

None of these blocks the immediate goal (unblocking the repo-wide red secret scan), but 2 and 3 are cheap and directly strengthen the guardrails this PR is built around, and 1 is a real scope question on a security control.

Recommendation

Mergeable as-is to unblock the scan; I'd fold in the two test-hardening fixes (2, 3) and make an explicit accept/scope decision on (1) before or right after merge. Merge itself targets protected main and is left for human sign-off — I'm not auto-merging, and I did not push to this branch (it isn't my designated working branch).


Generated by Claude Code

Copy link
Copy Markdown
Owner Author

Automated PR-remediation run — terminal state: HALTED(agent_lock_trusted_publication_missing)

The gitleaks fix itself is verified: gitleaks (working tree) passes on this head, and the substantive CI (test/build/lint/security scans/CodeQL/trivy) is green.

Three governance checks were red on open. I remediated the two that were body-structure failures by restructuring the PR description into the canonical pull_request_template.md sections (Canonical issueCloses #1158, Outcome, Scope, Risk, Verification, Production evidence, Agent handoff), keeping all of the original technical content and evidence — no facts changed, no fabricated evidence. Result:

  • PR Governance — now passing
  • Canonical issue and evidence — now passing
  • Agent completion enforcement — still failing: missing_trusted_publication

The remaining failure is not fixable by editing this PR. agent_completion_enforcement.py requires a check-run named Agent Lock trusted publication on the head SHA, published by a trusted GitHub App from .github/agent-lock/trusted-publishers.json. That is a trust-attestation control; an automated run cannot (and should not) forge it. It clears when the trusted publisher runs, or when a human decides the gate does not apply to this human-authored PR.

No merge was performed: protected main is human-gated and this run holds no authorization to bypass that. Next step is yours — either let the trusted publisher attest this head, or merge with the appropriate override once you're satisfied.


Generated by Claude Code

@groupthinking
groupthinking marked this pull request as draft August 1, 2026 20:16
@groupthinking groupthinking added duplicate This issue or pull request already exists security needs-triage labels Aug 1, 2026 — with ChatGPT Codex Connector

Copy link
Copy Markdown
Owner Author

Blocker-watch reconciliation at exact head 3bdc33e55b2162eb7b6badbeb442df57e46fba3e.

This PR is a competing implementation of the existing canonical Gitleaks repair in #1142 at exact head d85db13e1ae70c48e8e184d799579c9a4cdd610e.

Why #1142 remains canonical:

  • its final diff is one exact fingerprint in .gitleaksignore: uv.lock:square-access-token:5129;
  • exact-head CI, Coverage, CodeQL, Security, Secret Scan, and Dependency Review pass;
  • CodeRabbit and Copilot approved that exact head;
  • all inline threads are resolved.

Why this PR cannot advance:

  • three current unresolved Copilot threads remain: PRRT_kwDORAYbZs6Vqmou, PRRT_kwDORAYbZs6Vqmo2, and PRRT_kwDORAYbZs6Vqmo6;
  • the top-level line allowlist can suppress findings from unrelated rules when a PyPI URL is present;
  • its test assertions do not prove path regexes cannot exclude uv.lock or that the host boundary rejects foreign hosts.

The PR is closed unmerged as a duplicate. The branch is preserved; no commit or branch was deleted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

duplicate This issue or pull request already exists needs-triage python security

Projects

None yet

Development

Successfully merging this pull request may close these issues.

gitleaks (working tree) fails on every PR: uv.lock PyPI content hash trips square-access-token

2 participants